Plugin Networks for Inference under Partial Evidence

In this paper, we propose a novel method to incorporate partial evidence in the inference of deep convolutional neural networks. Contrary to the existing, top performing methods, which either iteratively modify the input of the network or exploit external label taxonomy to take the partial evidence into account, we add separate network modules ("Plugin Networks") to the intermediate layers of a pre-trained convolutional network. The goal of these modules is to incorporate additional signal, ie information about known labels, into the inference procedure and adjust the predicted output accordingly. Since the attached plugins have a simple structure, consisting of only fully connected layers, we drastically reduced the computational cost of training and inference. At the same time, the proposed architecture allows to propagate information about known labels directly to the intermediate layers to improve the final representation. Extensive evaluation of the proposed method confirms that our Plugin Networks outperform the state-of-the-art in a variety of tasks, including scene categorization, multi-label image annotation, and semantic segmentation.Comment: Accepted to WACV 202

Relative pointing offset analysis of calibration targets with repeated observations with Herschel-SPIRE Fourier-Transform Spectrometer

We present a method to derive the relative pointing offsets for SPIRE Fourier-Transform Spectrometer (FTS) solar system object (SSO) calibration targets, which were observed regularly throughout the Herschel mission. We construct ratios of the spectra for all observations of a given source with respect to a reference. The reference observation is selected iteratively to be the one with the highest observed continuum. Assuming that any pointing offset leads to an overall shift of the continuum level, then these ratios represent the relative flux loss due to mispointing. The mispointing effects are more pronounced for a smaller beam, so we consider only the FTS short wavelength array (SSW, 958-1546 GHz) to derive a pointing correction. We obtain the relative pointing offset by comparing the ratio to a grid of expected losses for a model source at different distances from the centre of the beam, under the assumption that the SSW FTS beam can be well approximated by a Gaussian. In order to avoid dependency on the point source flux conversion, which uses a particular observation of Uranus, we use extended source flux calibrated spectra to construct the ratios for the SSOs. In order to account for continuum variability, due to the changing distance from the Herschel telescope, the SSO ratios are normalised by the expected model ratios for the corresponding observing epoch. We confirm the accuracy of the derived pointing offset by comparing the results with a number of control observations, where the actual pointing of Herschel is known with good precision. Using the method we derived pointing offsets for repeated observations of Uranus (including observations centred on off-axis detectors), Neptune, Ceres and NGC7027. The results are used to validate and improve the point-source flux calibration of the FTS.Comment: 17 pages, 19 figures, accepted for publication in Experimental Astronom

The Liver Tumor Segmentation Benchmark (LiTS)

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LITS) organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2016 and International Conference On Medical Image Computing Computer Assisted Intervention (MICCAI) 2017. Twenty four valid state-of-the-art liver and liver tumor segmentation algorithms were applied to a set of 131 computed tomography (CT) volumes with different types of tumor contrast levels (hyper-/hypo-intense), abnormalities in tissues (metastasectomie) size and varying amount of lesions. The submitted algorithms have been tested on 70 undisclosed volumes. The dataset is created in collaboration with seven hospitals and research institutions and manually reviewed by independent three radiologists. We found that not a single algorithm performed best for liver and tumors. The best liver segmentation algorithm achieved a Dice score of 0.96(MICCAI) whereas for tumor segmentation the best algorithm evaluated at 0.67(ISBI) and 0.70(MICCAI). The LITS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.Comment: conferenc

Deep Learning Segmentation Algorithms for X-ray CT data

The segmentation task for 3D objects from X-ray CT volumetric data is of great significance for both industrial and medical applications. Deep learning techniques are narrowing the gap between human and machine capabilities in image segmentation. In this thesis we develop and discuss machine and deep learning techniques for semantic and instance segmentation. The techniques are evaluated on a dataset of CT scans of short glass fiber reinforced polymers prepared in cooperation with the University of Padova and on publicly available medical CT scans of lungs and liver. In addition to that, the last chapter is evaluated on a public and popular large-scale object detection, segmentation, and captioning dataset for a better comparison with the state-of-the-art. The chapters are structured in the following way: In chapter 2 we explain the short glass fiber reinforced polymer data acquisition together with the reference setup for quantitative comparison of segmentation techniques. The data creation process involves parts manufacturing, CT scanning, CT simulation, computational model design, volume reconstruction and ground-truth preparation. The reference setup consist of metrics for instance and semantic segmentation tasks as well as of a baseline, Frangi vesselness method. In chapter 3 we present a first deep learning model for semantic segmentation of fibers from CT scans. The model outperforms all the other methods including feature-engineered and machine learning models. In chapter 4 we present a first deep learning model for instance segmentation of fibers from CT scans. The model outperforms the state-of-the-art by a significant margin and is arguably the first method which allows calculation of important fiber statistics based on single-fiber segmentation. The model consist of a fully convolutional branch for semantic segmentation, and an enhanced branch for instance segmentation via proposed embedding learning loss function. In chapter 5 we present our work on use of machine learning techniques for medical CT analysis. We use a dictionary learning model and extend it to a 3D for bronchial vessels segmentation from thorax CT scans. Then, we discuss and develop a fully convolutional deep learning model for the task of liver and liver lesion segmentation from liver CT scans. Lastly, we present the Mask Mining training approach for boosting the semantic segmentation machine learning models. In chapter 6 we present the idea of the Plugin Networks as a solution for inference under partial evidence. The proposed framework can generalize to a number of machine learning tasks and is evaluated on the task of hierarchical scene categorization, multi-label image annotation and scene semantic segmentation achieving state-of-the-art on each